A Variational Autoencoder Cascade Generative Adversarial Network for Scalable 3D Object Generation and Reconstruction.

Generative Adversarial Networks (GANs) for 3D volume generation and reconstruction, such as shape generation, visualization, automated design, real-time simulation, and research applications, are receiving increased amounts of attention in various fields. However, challenges such as limited training data, high computational costs, and mode collapse issues persist. We propose combining a Variational Autoencoder (VAE) and a GAN to uncover enhanced 3D structures and introduce a stable and scalable progressive growth approach for generating and reconstructing intricate voxel-based 3D shapes. The cascade-structured network involves a generator and discriminator, starting with small voxel sizes and incrementally adding layers, while subsequently supervising the discriminator with ground-truth labels in each newly added layer to model a broader voxel space. Our method enhances the convergence speed and improves the quality of the generated 3D models through stable growth, thereby facilitating an accurate representation of intricate voxel-level details. Through comparative experiments with existing methods, we demonstrate the effectiveness of our approach in evaluating voxel quality, variations, and diversity. The generated models exhibit improved accuracy in 3D evaluation metrics and visual quality, making them valuable across various fields, including virtual reality, the metaverse, and gaming.

Optimization of Deep Learning Parameters for Magneto-Impedance Sensor in Metal Detection and Classification.

Deep learning technology is generally applied to analyze periodic data, such as the data of electromyography (EMG) and acoustic signals. Conversely, its accuracy is compromised when applied to the anomalous and irregular nature of the data obtained using a magneto-impedance (MI) sensor. Thus, we propose and analyze a deep learning model based on recurrent neural networks (RNNs) optimized for the MI sensor, such that it can detect and classify data that are relatively irregular and diverse compared to the EMG and acoustic signals. Our proposed method combines the long short-term memory (LSTM) and gated recurrent unit (GRU) models to detect and classify metal objects from signals acquired by an MI sensor. First, we configured various layers used in RNN with a basic model structure and tested the performance of each layer type. In addition, we succeeded in increasing the accuracy by processing the sequence length of the input data and performing additional work in the prediction process. An MI sensor acquires data in a non-contact mode; therefore, the proposed deep learning approach can be applied to drone control, electronic maps, geomagnetic measurement, autonomous driving, and foreign object detection.

Hybrid Bionic Nerve Interface for Application in Bionic Limbs.

Intuitive and perceptual neuroprosthetic systems require a high degree of neural control and a variety of sensory feedback, but reliable neural interfaces for long-term use that maintain their functionality are limited. Here, a novel hybrid bionic interface is presented, fabricated by integrating a biological interface (regenerative peripheral nerve interface (RPNI)) and a peripheral neural interface to enhance the neural interface performance between a nerve and bionic limbs. This interface utilizes a shape memory polymer buckle that can be easily implanted on a severed nerve and make contact with both the nerve and the muscle graft after RPNI formation. It is demonstrated that this interface can simultaneously record different signal information via the RPNI and the nerve, as well as stimulate them separately, inducing different responses. Furthermore, it is shown that this interface can record naturally evoked signals from a walking rabbit and use them to control a robotic leg. The long-term functionality and biocompatibility of this interface in rabbits are evaluated for up to 29 weeks, confirming its promising potential for enhancing prosthetic control.

Artificial intelligence-based numerical analysis of the quality of facial reanimation: A comparative retrospective cohort study between one-stage dual innervation and single innervation.

This study aimed to investigate the difference in facial reanimation surgery using functional gracilis muscle transfer between the masseteric nerve alone and its combined use with cross face nerve graft (CFNG), which has not been explored before. A novel analysis method based on artificial intelligence (AI) was employed to compare the outcomes of the two approaches. Using AI, 3-dimensional facial landmarks were extracted from 2-dimensional photographs, and distance and angular symmetry scores were calculated. The patients were divided into two groups, with Group 1 undergoing one-stage CFNG and masseteric nerve dual innervation, and Group 2 receiving only masseteric nerve. The symmetry scores were obtained before and 1 year after surgery to assess the degree of change. Of the 35 patients, Group 1 included 13 patients, and Group 2 included 22 patients. The analysis revealed that, in the resting state, the change in the symmetry score of the mouth corner showed distance symmetry (2.55 ± 2.94, 0.52 ± 2.75 for Group 1 and Group 2, respectively, p = 0.048) and angle symmetry (1.21 ± 1.43, 0.02 ± 0.22 for Group 1 and Group 2, respectively, p = 0.001), which were significantly improved in Group 1, indicating a more symmetric pattern after surgery. In the smile state, only the angle symmetry was improved more symmetrically in Group 1 (3.20 ± 2.38, 1.49 ± 2.22 for Group 1 and Group 2, respectively, p = 0.041). Within the limitations of the study it seems that this new analysis method enabled a more accurate numerical symmetry score to be obtained, and while the degree of mouth corner excursion was sufficient with only the masseteric nerve, accompanying CFNG led to further improvement in symmetry in the resting state.

Graph Convolutional Network for 3D Object Pose Estimation in a Point Cloud.

Graph Neural Networks (GNNs) are neural networks that learn the representation of nodes and associated edges that connect it to every other node while maintaining graph representation. Graph Convolutional Neural Networks (GCNs), as a representative method in GNNs, in the context of computer vision, utilize conventional Convolutional Neural Networks (CNNs) to process data supported by graphs. This paper proposes a one-stage GCN approach for 3D object detection and poses estimation by structuring non-linearly distributed points of a graph. Our network provides the required details to analyze, generate and estimate bounding boxes by spatially structuring the input data into graphs. Our method proposes a keypoint attention mechanism that aggregates the relative features between each point to estimate the category and pose of the object to which the vertices of the graph belong, and also designs nine degrees of freedom of multi-object pose estimation. In addition, to avoid gimbal lock in 3D space, we use quaternion rotation, instead of Euler angle. Experimental results showed that memory usage and efficiency could be improved by aggregating point features from the point cloud and their neighbors in a graph structure. Overall, the system achieved comparable performance against state-of-the-art systems.

Uniformity improvement of a reconstructed-holographic image in a near-eye display system using off-axis HOE.

When a near-eye display (NED) device reproduces an image at a location close to the eye, the virtual image is implemented at a large angle. The uniformity of the image is unbalanced due to the change in diffraction efficiency by the hologram recording angle and angular selectivity. This study proposes a method for implementing an optimal uniform image by analyzing the diffraction efficiency and the reconstructed image was analyzed using angular selectivity generated while reproducing the source point of the diffused image as an intermediate element by holographic optical element (HOE). This research provides practical results for displaying high diffraction efficiency and immersive holographic images in the NED system with HOE as uniformed intermediate elements.

Numerical Approach to Facial Palsy Using a Novel Registration Method with 3D Facial Landmark.

Treatment of facial palsy is essential because neglecting this disorder can lead to serious sequelae and further damage. For an objective evaluation and consistent rehabilitation training program of facial palsy patients, a clinician's evaluation must be simultaneously performed alongside quantitative evaluation. Recent research has evaluated facial palsy using 68 facial landmarks as features. However, facial palsy has numerous features, whereas existing studies use relatively few landmarks; moreover, they do not confirm the degree of improvement in the patient. In addition, as the face of a normal person is not perfectly symmetrical, it must be compared with previous images taken at a different time. Therefore, we introduce three methods to numerically approach measuring the degree of facial palsy after extracting 478 3D facial landmarks from 2D RGB images taken at different times. The proposed numerical approach performs registration to compare the same facial palsy patients at different times. We scale landmarks by performing scale matching before global registration. After scale matching, coarse registration is performed with global registration. Point-to-plane ICP is performed using the transformation matrix obtained from global registration as the initial matrix. After registration, the distance symmetry, angular symmetry, and amount of landmark movement are calculated for the left and right sides of the face. The degree of facial palsy at a certain point in time can be approached numerically and can be compared with the degree of palsy at other times. For the same facial expressions, the degree of facial palsy at different times can be measured through distance and angle symmetry. For different facial expressions, the simultaneous degree of facial palsy in the left and right sides can be compared through the amount of landmark movement. Through experiments, the proposed method was tested using the facial palsy patient database at different times. The experiments involved clinicians and confirmed that using the proposed numerical approach can help assess the progression of facial palsy.

Change in Blink Rate in the Metaverse VR HMD and AR Glasses Environment.

Blink rate, a major physiological response in humans, directly affects ocular diseases such as keratitis and dry eye. The blink rate in normal eyes appears at a constant frequency of 6-30 times per minute and is constant for each individual. In a previous study, the blink rate decreased when viewing content with high intensity and realism. Therefore, we tried to investigate the change in blink rate when viewing the content in VR HMD (virtual reality head-mounted display) and AR (augmented reality) glasses environments. We compared and analyzed the blink rate in four environments: natural state, viewing monitor, viewing VR HMD, and viewing AR glasses. Twenty-one participants (age, 26.87 ± 3.31 years) viewed the content for 1 min in four environments. One-way repeated ANOVA was used to analyze the blink rate changes. The study showed that the blink rate was decreased in the monitor, VR HMD, and AR glasses environments compared to that in the natural environment. Comparing the VR HMD environment with the AR glasses environment showed that the blink rate decreased in the VR HMD environment. The results of this study can be used for content use safety recommendations (guidelines for safe use of contents due to decreased blink rate) in the VR HMD and AR glasses environments, which are currently attracting attention in the metaverse.

Improved Feature-Based Gaze Estimation Using Self-Attention Module and Synthetic Eye Images.

Gaze is an excellent indicator and has utility in that it can express interest or intention and the condition of an object. Recent deep-learning methods are mainly appearance-based methods that estimate gaze based on a simple regression from entire face and eye images. However, sometimes, this method does not give satisfactory results for gaze estimations in low-resolution and noisy images obtained in unconstrained real-world settings (e.g., places with severe lighting changes). In this study, we propose a method that estimates gaze by detecting eye region landmarks through a single eye image; and this approach is shown to be competitive with recent appearance-based methods. Our approach acquires rich information by extracting more landmarks and including iris and eye edges, similar to the existing feature-based methods. To acquire strong features even at low resolutions, we used the HRNet backbone network to learn representations of images at various resolutions. Furthermore, we used the self-attention module CBAM to obtain a refined feature map with better spatial information, which enhanced the robustness to noisy inputs, thereby yielding a performance of a 3.18% landmark localization error, a 4% improvement over the existing error and A large number of landmarks were acquired and used as inputs for a lightweight neural network to estimate the gaze. We conducted a within-datasets evaluation on the MPIIGaze, which was obtained in a natural environment and achieved a state-of-the-art performance of 4.32 degrees, a 6% improvement over the existing performance.

A Novel Method Based on GAN Using a Segmentation Module for Oligodendroglioma Pathological Image Generation.

Digital pathology analysis using deep learning has been the subject of several studies. As with other medical data, pathological data are not easily obtained. Because deep learning-based image analysis requires large amounts of data, augmentation techniques are used to increase the size of pathological datasets. This study proposes a novel method for synthesizing brain tumor pathology data using a generative model. For image synthesis, we used embedding features extracted from a segmentation module in a general generative model. We also introduce a simple solution for training a segmentation model in an environment in which the masked label of the training dataset is not supplied. As a result of this experiment, the proposed method did not make great progress in quantitative metrics but showed improved results in the confusion rate of more than 70 subjects and the quality of the visual output.

Prediction of Wave Transmission Characteristics of Low-Crested Structures with Comprehensive Analysis of Machine Learning.

The adoption of low-crested and submerged structures (LCS) reduces the wave behind a structure, depending on the changes in the freeboard, and induces stable waves in the offshore. We aimed to estimate the wave transmission coefficient behind LCS structures to determine the feasible characteristics of wave mitigation. In addition, various empirical formulas based on regression analysis were proposed to quantitatively predict wave attenuation characteristics for field applications. However, inherent variability of wave attenuation causes the limitation of linear statistical approaches, such as linear regression analysis. Herein, to develop an optimization model for the hydrodynamic behavior of the LCS, we performed a comprehensive analysis of 10 types of machine learning models, which were compared and reviewed on the prediction accuracy with existing empirical formulas. We found that, among the 10 models, the gradient boosting model showed the highest prediction accuracy with MSE of 1.0 × 10-3, an index of agreement of 0.996, a scatter index of 0.065, and a correlation coefficient of 0.983, which indicates a performance improvement over the existing empirical formulas. In addition, based on a variable importance analysis using explainable artificial intelligence, we determined the significant importance of the input variable for the relative freeboard (RC/H0) and the relative freeboard to water depth ratio (RC/h), which confirms that the relative freeboard was the most dominant factor for influencing wave attenuation in the hydraulic behavior around the LCS. Thus, we concluded that the performance prediction method using a machine learning model can be applied to various predictive studies in the field of coastal engineering, deviating from existing empirical-based research.

Efficient Single-Shot Multi-Object Tracking for Vehicles in Traffic Scenarios.

Multi-object tracking is a significant field in computer vision since it provides essential information for video surveillance and analysis. Several different deep learning-based approaches have been developed to improve the performance of multi-object tracking by applying the most accurate and efficient combinations of object detection models and appearance embedding extraction models. However, two-stage methods show a low inference speed since the embedding extraction can only be performed at the end of the object detection. To alleviate this problem, single-shot methods, which simultaneously perform object detection and embedding extraction, have been developed and have drastically improved the inference speed. However, there is a trade-off between accuracy and efficiency. Therefore, this study proposes an enhanced single-shot multi-object tracking system that displays improved accuracy while maintaining a high inference speed. With a strong feature extraction and fusion, the object detection of our model achieves an AP score of 69.93% on the UA-DETRAC dataset and outperforms previous state-of-the-art methods, such as FairMOT and JDE. Based on the improved object detection performance, our multi-object tracking system achieves a MOTA score of 68.5% and a PR-MOTA score of 24.5% on the same dataset, also surpassing the previous state-of-the-art trackers.

Neural Network for Metal Detection Based on Magnetic Impedance Sensor.

The efficiency of the metal detection method using deep learning with data obtained from multiple magnetic impedance (MI) sensors was investigated. The MI sensor is a passive sensor that detects metal objects and magnetic field changes. However, when detecting a metal object, the amount of change in the magnetic field caused by the metal is small and unstable with noise. Consequently, there is a limit to the detectable distance. To effectively detect and analyze this distance, a method using deep learning was applied. The detection performances of a convolutional neural network (CNN) and a recurrent neural network (RNN) were compared from the data extracted from a self-impedance sensor. The RNN model showed better performance than the CNN model. However, in the shallow stage, the CNN model was superior compared to the RNN model. The performance of a deep-learning-based (DLB) metal detection network using multiple MI sensors was compared and analyzed. The network was detected using long short-term memory and CNN. The performance was compared according to the number of layers and the size of the metal sheet. The results are expected to contribute to sensor-based DLB detection technology.

NRA-Net-Neg-Region Attention Network for Salient Object Detection with Gaze Tracking.

In this paper, we propose a detection method for salient objects whose eyes are focused on gaze tracking; this method does not require a device in a single image. A network was constructed using Neg-Region Attention (NRA), which predicts objects with a concentrated line of sight using deep learning techniques. The existing deep learning-based method has an autoencoder structure, which causes feature loss during the encoding process of compressing and extracting features from the image and the decoding process of expanding and restoring. As a result, a feature loss occurs in the area of the object from the detection results, or another area is detected as an object. The proposed method, that is, NRA, can be used for reducing feature loss and emphasizing object areas with encoders. After separating positive and negative regions using the exponential linear unit activation function, converted attention was performed for each region. The attention method provided without using the backbone network emphasized the object area and suppressed the background area. In the experimental results, the proposed method showed higher detection results than the conventional methods.

Robust Extrinsic Calibration of Multiple RGB-D Cameras with Body Tracking and Feature Matching.

RGB-D cameras have been commercialized, and many applications using them have been proposed. In this paper, we propose a robust registration method of multiple RGB-D cameras. We use a human body tracking system provided by Azure Kinect SDK to estimate a coarse global registration between cameras. As this coarse global registration has some error, we refine it using feature matching. However, the matched feature pairs include mismatches, hindering good performance. Therefore, we propose a registration refinement procedure that removes these mismatches and uses the global registration. In an experiment, the ratio of inliers among the matched features is greater than 95% for all tested feature matchers. Thus, we experimentally confirm that mismatches can be eliminated via the proposed method even in difficult situations and that a more precise global registration of RGB-D cameras can be obtained.

Substrate Effect of Platinum-Decorated Carbon on Enhanced Hydrogen Oxidation in PEMFC.

Environmentally sustainable fuel cells with high efficiency have attracted much attention as a promising approach to resolving future energy problems. However, some obstacles must be overcome, such as corrosion, water control, and long-term degradation. Herein, we investigated the improved electrochemical performance and hydrogen oxidation reaction (HOR) mechanism of platinum loaded on carbon nanotube (Pt/CNT) catalyst by conducting experimental and theoretical studies. The Pt/CNT catalyst had a larger active area than the Pt/C (platinum loaded on carbon black) catalyst and also exhibited improved performance due to its long-term stability. In addition, the charge-transfer resistance of Pt/CNT (61.2 Ω cm2) is much smaller than that of Pt/C (90.2 Ω cm2), indicating that the CNT support offers good electron transfer. To further understand the hydrogen dissociation mechanisms of Pt/CNT and Pt/C, we investigated the adsorption characteristics and electron transfer of the catalysts with optimized geometry using the density functional theory (DFT). Pt/CNT exhibited higher adsorption energy and electron transfer than Pt/C, which leads to improved HOR. The integrated experimental and theoretical study conducted here suggests that Pt/CNT is a promising candidate for maintaining the performance of cathode catalysts in the polymer electrolyte membrane fuel cell.

Development of Real-Time Hand Gesture Recognition for Tabletop Holographic Display Interaction Using Azure Kinect.

The use of human gesturing to interact with devices such as computers or smartphones has presented several problems. This form of interaction relies on gesture interaction technology such as Leap Motion from Leap Motion, Inc, which enables humans to use hand gestures to interact with a computer. The technology has excellent hand detection performance, and even allows simple games to be played using gestures. Another example is the contactless use of a smartphone to take a photograph by simply folding and opening the palm. Research on interaction with other devices via hand gestures is in progress. Similarly, studies on the creation of a hologram display from objects that actually exist are also underway. We propose a hand gesture recognition system that can control the Tabletop holographic display based on an actual object. The depth image obtained using the latest Time-of-Flight based depth camera Azure Kinect is used to obtain information about the hand and hand joints by using the deep-learning model CrossInfoNet. Using this information, we developed a real time system that defines and recognizes gestures indicating left, right, up, and down basic rotation, and zoom in, zoom out, and continuous rotation to the left and right.

An Extended Method for Saccadic Eye Movement Measurements Using a Head-Mounted Display.

Saccadic eye movementis an important ability in our daily life and is especially important in driving and sports. Traditionally, the Developmental Eye Movement (DEM) test and the King-Devick (K-D) test have been used to measure saccadic eye movement, but these only involve measurements with "adjusted time". Therefore, a different approach is required to obtain the eye movement speed and reaction rate in detail, as some are rapid eye movements, while others are slow actions, and vice versa. This study proposed a extended method that can acquire the "rest time" and "transfer time", as well as the "adjusted time", by implementing a virtual reality-based DEM test, using a FOVE virtual reality (VR) head-mounted display, equipped with an eye-tracking module. This approach was tested in 30 subjects with normal vision and no ophthalmologic disease by using a 2-diopter (50-cm) distance. This allowed for measurement of the "adjusted time" and the "rest time" for focusing on each target number character, the "transfer time" for moving to the next target number character, and recording of the gaze-tracking log. The results of this experiment showed that it was possible to analyze more parameters of the saccadic eye movement with the proposed method than with the traditional methods.

Real-Time Vehicle-Detection Method in Bird-View Unmanned-Aerial-Vehicle Imagery.

Vehicle detection is an important research area that provides background information for the diversity of unmanned-aerial-vehicle (UAV) applications. In this paper, we propose a vehicle-detection method using a convolutional-neural-network (CNN)-based object detector. We design our method, DRFBNet300, with a Deeper Receptive Field Block (DRFB) module that enhances the expressiveness of feature maps to detect small objects in the UAV imagery. We also propose the UAV-cars dataset that includes the composition and angular distortion of vehicles in UAV imagery to train our DRFBNet300. Lastly, we propose a Split Image Processing (SIP) method to improve the accuracy of the detection model. Our DRFBNet300 achieves 21 mAP with 45 FPS in the MS COCO metric, which is the highest score compared to other lightweight single-stage methods running in real time. In addition, DRFBNet300, trained on the UAV-cars dataset, obtains the highest AP score at altitudes of 20-50 m. The gap of accuracy improvement by applying the SIP method became larger when the altitude increases. The DRFBNet300 trained on the UAV-cars dataset with SIP method operates at 33 FPS, enabling real-time vehicle detection.

Mg-Ion Inversion in MgO@MgO-Al2 O3 Oxides: The Origin of Basic Sites.

Although MgO-Al2 O3 is well known as having a spinel structure, the inversion of which occurs by exchange of the trivalent (Al3+ ) and divalent (Mg2+ ) cations, little analytical study of the degree of inversion has been carried out. This study concerns a simple methodology to identify the inversion by solid-state NMR spectroscopy, whereby its correlation with the CO2 capture capacity of MgO-rich MgO@MgO-Al2 O3 spinel structures is verified. Through 27 Al and 25 Mg NMR spectroscopy, temperature-programmed CO2 desorption, and thermogravimetric analysis, higher inversion is found to occur at low Mg/Al ratios and the inversion is found to decrease as the Mg/Al ratio increases. Moreover, the degree of inversion correlates with CO2 sorption, which is associated with the medium-strength basic sites induced by formation of the unsaturated O2- species. These results will open new pathways to exploit defects in complex oxides beyond spinels and their derivatives for desired applications. This demonstration of MgO-Al2 O3 for CO2 sorption can contribute to the design of future CO2 sorbents.